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ABSTRACT 


^A  school  may  be  viewed  as  an  enterprise  in  which  the  professional 
staff  provide  the  operating  conditions  for  converting  quantifiable 
resources  or  inputs  into  pupil  learning  (outputs) .  The  resources  are 
determined  by  budgets,  teacher  assignments,  and  student  assignments 
while  learning  is  determined  by  various  outputs  scored  according  to 
standardized  tests  such  as  the  Iowa  Test .  of  Basic  Skills.  Following 
the  work  of  Charnes,  Cooper,  and  Rhodes  '■j'tlj  ,  we  use  a  ratio  definition 
of  efficiency  which  takes  account  of  all  ouputs  and  inputs  without 
requiring  a  priori  specification  of  weights.  Instead  a  series  of  , 
mathematical  programs  are  applied  to  determine  Vvirtual  multipliers'^ 
.from  actual  data  which  yield  the  values  that  can  be  regarded  as  the 
Tmost  favorable  weights^  for  each  school  being  evaluated.  If  the 
resulting  optimum  virtual  multipliers  for  a  given  school  yields  an 
efficiency  ratio  of  one,  then  that  school  is  said  to  be  efficient. 

If  the  ratio  is  less  than  one  then  that  school  is  said  to  be  inefficient 
relative  to  the  other  schools  in  the  analysis.  The  ratio  is  also 
accorded  operational  significance — it  is  not  merely  an  index  number— 
so  that  the  resulting  values  and  the  associated  virtual  multipliers 
make  it  possible  to  locate  where  improvements  may  be  made  along  with 
their  relative  magnitudes. 

This  analysis  was  applied  to  167  elementary  schools  in  the 
Houston  Independent  School  District.,  Of  these  schools,  78  were 
found  to  be  inefficiently  utilizing  their  resources  as  compared  to 


the  89  efficiant  schools.  Based  on  this  pilot  study,  an  Educational  j 
Productivity  Council  has  been  fotiv.ed  at  the  University  of  Texas  at  t 
Austin  to  provide  an  annual  analysis  for  all  of  its  member  schools.  j 
At  present  285  Texa*  schools  in  22  districts  are  scheduled  for  j 
participation  in  the  annual  analysis  as  described  in  this  investi-  ■ 
gation.  3 
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I.  INTRODUCTION 


In  the  decade  of  the  197C’s  a  pervasive  push  for  account¬ 
ability  in  public  education  in  the  United  States  [22]  brought 
about  revolutionary  changes  in  the  management  problems  faced  by 
administrators  of  schools.  These  forces  came  to  bear  with  swift 
and  urgent  impact  on  large  metropolitan  school  districts  such  as 
the  Houston  Independent  School  District,  a  complex  organization 
with  16,000  employees,  194,000  pupils,  and  an  annual  budget  of 
$320  million.  Rapid  population  growth,  federal  regulatory  agency 
mandates  and  court  orders  implementing  civil  rights  legislation 
[ I5X,  combined  with  declining  birth  rates  and  alleged  white  flight 
to  suburban  schools,  created  a  set  of  problems  for  which  few 
manangement  science  techniques  have  been  developed  and  tested. 

The  one  that  the  Board  of  Education  felt  was  most  important,  and  that 
provided  the  occasion  for  the  research  reported  here,  was  the  need 
for  management  information  concerning  the  relative  productivity  of 
the  241  individual  schools  in  relation  to  the  resources  allocated 
and  the  environmental  factors  affecting  their  outputs. 

For  five  years,  1975-1979,  the  administration  published  an 
annual  report  of  achievement  scores  for  each  school  along  with 
input  data  such  as  class  size,  professional  staff  characteristics, 
per  pupil  expenditures,  ethnic  group  enrollment,  and  socio¬ 
economic  status  1 14] .  However,  no  attempt  was  made  to  relate 
these  inputs  to  the  outputs  of  each  school.  Then  in  May  1979,  in 
an  effort  to  increase  public  confidence  in  Houston  public  schools, 


the  Board  of  Education  published  the  Comparison  of  Academic 
Performance  Data  for  Students  in  the  Houston  Independent  School 
District  and  Other  Public  School  Systems  [29J  .  This  was  a  bold  and 
unprecedented  step,  but  the  report  lacked  a  comprehensive  model  for 
making  the  comparisons  required  across  the  wide  ranges  of  output 
and  input  mixes  and  levels  involved,  relying  instead  upon  a  simple 
comparison  of  mean  achievement  in  schools,  controlling  for  socio¬ 
economic  level. 

The  need  for  a  better  quantitative  model  became  imperative 
later  that  year  when  the  state  legislature  adopted  a  law  requiring 
statewide  competency  testing  of  pupils  in  all  Texas  schools  and 
stipulated  that  "...student  performance  data  shall  be  aggregated 
by  campus  and  district  and  made  available  to  the  public,  with 
appropriate  interpretations..."  [28^. 

1.1  Problem  Description 

Out  of  the  context  described  above,  the  analysis  on  this 
study  sought  to  provide  a  way  to  obtain  a  valid  efficiency  measure 
for  each  school  that  would  be  empirically  based  and  logically 
justified  and  would  also  provide  a  measure  of  each  school's 
ability  to  produce  desired  outputs  from  their  valued  inputs.  It 
was  also  desired  to  compute  the  extent  to  which  inputs  were  non¬ 
productive  in  inefficient  school  units  and,  if  possible,  to  go 
even  further  and  estimate  the  augmentations  in  outputs  and/or 
the  reductions  in  inputs  that  could  be  attained  if  efficiency  were 


to  be  achieved 


Taken  all  together,  then,  a  solution  to  this  problem  would 
provide  management  with  information  like  the  following: 

1)  An  efficiency  evaluation  of  individual  schools  which  would 
include  the  productivity  of  the  professional  staff  of  the 
school  while  making  allowance  for  the  conditions  under  which 
they  were  operating. 

2)  Targeted  output  goals  and  identification  of  needed  input 
modifications. 

3)  Identification  of  areas  in  which  efficiency  could  be  increased. 

1.2  Deficiencies  in  Prior  Methods  Employed 

We  have  been  unable  to  find  any  methods,  other  than  the  one 
used  in  this  investigation,  that  provide  an  overall  operational 
definition  of  the  efficiency  of  a  school.  There  is,  however,  a 
considerable  body  of  literature  documenting  attempts  to  define 
production  functions  in  education.  Levin  j^24j  and  Hanushek  [l6j 
characterize  current  methodologies  as  being  deficient  both  concept¬ 
ually  and  in  their  implementation.  Following  one  such  attempt. 

Levin  j^23j  concluded,  "The  analysis. .  .is  fraught  with  difficulties 
that  are  unusually  severe  given  the  present  analytical  state  of  the 
art." 

There  are  several  difficulties  which  are  immediately  apparent 
in  that  the  economic  theory  of  production  functions  requires 
them  to  be  extremal  estimates  whereas  the  statistical  methods 
employed  do  not  generally  conform  to  this  requirement.1  In  addition 
the  situation  of  multiple  outputs  is  not  adequately  dealt  with. 
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The  method  most  commonly  employed  is  least  squares  linear 
regression  with  a  single  output.  A  variant  method  involves  uses 
of  multiple  regressions — perhaps  in  repeated  forms — to  estimate 
the  effects  of  the  same  inputs  on  different  outputs.  Some  major 
criticisms  of  the  methods  that  have  previously  been  used  to  develop 
production  functions  may  be  summarized  as: 

(i)  Multiple  outputs  are  not  taken  into 

account  simultaneously.  See  Boardman, 

Davis  and  Sanday  [sj  who  use  econometric 
(simultaneous  relation)  estimates  to 
circumvent  this  difficulty  but  do  not 
deal  with  (or  even  discuss)  the  problem 
of  obtaining  extremal  estimates. 

(ii)  Regression  coefficients  do  not  necessarily 
indicate  the  most  efficient  way  to  produce 
an  impact  on  outputs  as  noted  by  Bowles  |Yj 
who  does  not,  however,  provide  any  guidance 
on  how  this  might  be  accomplished. 

(iii)  There  is  a  lack  of  agreement  even  on  the 

meaning  of  economic  terms  such  as  "technical 
efficiency"  vs.  other  kinds  of  efficiency 
when  applied  to  schools  (Levin  jj24j  )  . 

(iv)  The  interdependency  of  inputs  (multi - 

collinearity)  and  outputs,  too,  may  produce 
misleading  results  if  the  resulting  coefficients 
are  used  to  determine  alternatives  in  input 
mixes  or  levels  (Bowles  [7]  ,  Bowles  and  Levin 

H>- 
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(v)  The  output-input  relationships  may  not  be 
linear  or  independent  and,  moreover,  there 
are  no  guides  available  (theoretical  or 
otherwise)  for  determining  the  classes 
of  parametric  functional  forms  to  be  used 
in  these  statistical  estimation  models  (Levin 
[24]  ,  Bowles  [7]  )  . 

(vi)  None  of  the  production  function  studies  are 
based  upon  controlled  experiments  in  which 
inputs  have  been  manipulated  (Averch  et. 
al.  ["2J  ) .  Note,  however,  that  the  theory  of 
experimental  design  does  not  deal  at  all  with 
problems  involved  in  allowing  for  differences 
in  managerial  efficiency  and/or  other  such 
variables  that  enter  importantly  into  the 
resulting  outputs. 

The  approach  proposed  by  Charnes,  Cooper  and  Rhodes  [ll,12, 
circumvents  these  difficulties.  For  one  thing,  it  does  not 
require  the  production  functions  to  be  specified  in  parametric 
form  and,  indeed,  it  allows  for  production  functions  which  may 
differ  for  each  school  with  multiple  outputs  and  multiple  inputs 
that  may  be  related  to  each  other  in  numerous  ways  (linear  or 
nonlinear)  that  need  not  be  specified.  Furthermore,  the  resulting 
overall  (scalar)  measures  of  efficiency  are  obtained  from  extremal 
methods  which  relate  the  results  to  mathematical  programming  models 
in  which  all  outputs  are  explicity  identified.  The  values  assigned 
to  these  inputs  and  outputs,  which  are  referred  to  as  virtual 


multipliers,  help  to  locate  sources  of  inefficiency  on  the  one 
hand  and  also  to  indicate  tradeoffs  along  the  efficiency  frontiers 
for  additional  use  as  required.  How  this  is  accomplished  will  be 
made  clear,  at  least  in  part,  in  the  sections  that  follows. 


II.  MATHEMATICAL  MODEL 


Note  theu  k  appears  in  the  constraints  as  well  as  the  functional 


so  that,  automatically 

*  / 

(3)  maximum  h.  ■  h.  ==  1. 


Moreover,  all  y  .,  x,.  ">  0  with  u  ,  v,  0 

J  *2  3-2  r  i 

★ 

guarantees  that  h^  0  will  apply  and  a  solution  satisfying 

.  .  2 

these  conditions  always  exists. 

Drawing  all  of  the  above  conditions  together  we  then  say  that 

Ilf 

h^  »  1  if  and  only  if  school  k  is  efficient  relative  to  the  other 
schools  using  these  same  inputs  and  producing  these  same  outputs 
in  the  set  of  j  “  1,  2,  ...»  n  schools  being  considered. 

It  can  be  shown  that  this  development  can  be  related  to  the 
economic  condition  of  Pareto  optimality.3  Here,  however,  it 

will  suffice  to  observe  that  we  are  according  school  k  a  favored 

*  *  . 

position  in  that  the  optimal  ur»  S  0  are  selected  to  give  this 

school  the  highest  possible  efficiency  rating  that  the  data  will 

*  / 

allow.  Hence,  if  h^  %.  1  then  this  school  is  inefficient  and 
cannot  achieve  a  higher  rating  relative  to  the  reference  set 
with  which  it  is  being  compared. 

*  ^ 

As  we  shall  shortly  see  these  h^  V  1  values  can  be  accorded 
operational  significance  in  the  form  of  output  increases  or  input 
decreases.  For  the  present,  however,  we  may  observe  that  the 
above  ratio  form  of  the  model  inv^l^es  a  nonlinear  programming 
problem.  As  Charnes,  Cooper,  and  Rhodes  have  shown  in  [12J  , 
however,  it  may  be  replaced  by  an  ordinary  linear  programming 
problem. 
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The  latter  development  is  accomplished  by  means  of  the  theory 

of  linear  fractional  programming  as  gi'*en  in  £9]  .  We  do  not 

repeat  that  development  here,  however,  and  merely  replace  the 

4 

problem  (2)  by  the  following: 

m 


min  g, 
k 


3  v.  x., 
i  ik 

i  *  1 


subject  to 
m 


(3) 


0  I  ViXij  "  I  -r‘rj 

i  =  1  r  -  1 

s 

1  ■  X  \  vck 


u  y  .,  j  *  1,2,  • ■ • ,  n 


r  -  1 


£  v^  ,  i  *  1,2,  •••,  m 

^  ^  u^  ,  r  ■  1,2,  •••»  s 

We  then  call  g*  =  minimum  g  the  reciprocal  of  the  efficiency 

K 

*  1 

index  defined  by  h  = 

*  gk  . 


Since  the  number  of  schools  j  »  1,2,  . ..,  n  is  usually  much 
larger  than  the  number  of  inputs  and  outputs  to  be  considered,  we 
take  advantage  of  the  duality  relations  of  linear  programming  and 
replace  (3)  with  the  following: 
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A 


% 


4 


i 

i 


JrtVllflSliui  ftl  ftlti  il  KltV  l 


\j  s+r,  s.  ^  0  for  all  j,  r,  and  i. 

*  *  *  1 

At  an  optimum  we  have  z.  *  g.  so  that  h.  *  — *—  with  the 

K  K  K 

resulting  relations  showing  how  to  move  between  the  ratio  and  linear 

programming  forms  of  the  problems  as  required.  There  is  now 

+*  •>. 

available  extra  information,  however,  in  that  any  0  or 

-*  5 

s^  >  0  also  indicates  that  efficiency  has  not  been  achieved. 

*  w  +*  _*  v 

Noting  that  1  and/or  >  0  or  ^  0  represent 

sources  of  inefficiency  we  next  observe  that  efficiency  can  be 
attained  if  we  apply  these  results  to  the  original  data  in  the 
form 


A 

X., 

ik 

_* 

*  x . ,  -  s.  ,  i  = 

xk  x 

Xf2f  e  •  •  t  rn 

A 

*  4.* 

r  *  1  §  2  /  •  •  m  §  s  ■ 

yrk 

■  y  ,  z,  +  s  , 

■*  rk  k  r 

In  other  words  we  adjust  the  original  an^  yrk  observations  in 
the  manner  indicated  by  (5)  to  obtain  new  values  x^  ,  $  which 
would  render  school  k  efficient.  Note,  in  particular,  that  input 
reductions  and  output  augmentations  may  be  required  simultaneously. 

Computer  software  was  written  to  solve  n  linear  programming 
models  defined  in  (4)  in  a  single  run  via  a  new  general  purpose 
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linear  programming  code  for  the  modified  simplex  method.  For  each 


*_ 


*+ 


school  the  program  produces  the  optimal  z^»  h^,  s^  ,  s_  ,  u  ,  v  ,  x^,  ^ 
and  the  results  of  ranging  on  the  right-hand-side.  The  program  begins 


with  the  initial  feasible  solution 


1  and  all  other  variables 


k  k 

zero.  The  LP  code  maintains  the  basis  inverse  in  product  form  and 
only  the  non-zeros  are  stored.  The  LP  code  reinversion  routine 


is  based  on  the  work  of  Hellerman  and  Rarick 


[«]• 


This  routine 


X'.  j'-rporates  a  technique  known  as  "splitting  the  bump"  (see  Kalan  J^2o| 
and  Orchard-Hays  [25J )  and  uses  the  "spike  swapping  theory"  of 
Helgason  and  Kennington  £l7j  .  A  good  description  of  these 
computational  procedures  may  be  found  in  Helgason  £isj.  The 
arrays  and  working  files  require  approximately  (3n  +  13) (s  +  m)  +  4n 
words  of  core  storage.  This  package  for  efficiency  determination 
is  described  and  documented  in  [21J .  A  revised  package  with 
more  convenient  data  inputs  and  informational  print  out  of  solutions 
determines  efficiency  directly,  instead  of  its  reciprocal,  and 
is  presented  in  £l]  . 
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III.  SELECTION  OF  OUTPUTS  AND  INPUTS 

The  Iowa  Test  of  Basic  Skills  (ITBS),  a  comprehensive  standarized 
test  of  achievement,  had  been  administered  to  all  elementary  school 
pupils  in  Houston  in  May,  1978.  Among  other  measures,  the  ITBS 
provided  a  composite  score  for  the  aggregated  sub-test  scores  of  each 
pupil;  the  mean  of  the  composite  score  for  grade  three  and  the  mean 
for  grade  sax  were  used  as  the  two  output  measures  for  each  school. 

This  high  level  of  aggregation  —»  over  pupils  and  over  sub-tests  — 
seemed  adequate  for  the  purpose  of  the  study.  If  detailed  individual 
school  planning  information  were  required,  sub-test  scores  yould  be 
the  relevant  measures,  with  the  scores  aggregated  by  classroom*  It 
should  be  noted  also  that  available  data  restricted  the  analysis  to 
cognitive  outputs,  although  there  are  other  school  outputs  that  are  valued. 

The  twelve  input  measures  were  selected  from  those  that  had  been 
reported  by  the  Houston  administration  in  their  annual  Elementary  School 
Profiles  {14] .  These  included: 

(A)  M&uum  oh  thz  ckOAM.ZtZ/UAtic6  oh 

pupils  that  wznz  highly  c.oMzlatzd.  with 
achlzve/nznt  szoau: 

(1  and  2)  The  previous  year's  achievement 
scores  on  the  same  test  battery 
for  the  preceding  grade  levels  — 

2nd  and  5th  grades, 
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(3)  percent  non-minority  enrollment,6 

(4)  percent  of  students  paying  full  lunch 
price,  and 

(5)  percent  attendance:  average  dally  attendance 
*  average  dally  membership 

(B)  MuuuAt 6  oi  school  A.&60UA cti  available.} 

(6)  number  of  professionals  per  100 
pupils, 

(7)  local  and  state  expenditures  per 
pupil, 

(8)  federal  money  allocated  per  pupil, 

(9)  number  of  special  programs  In  the 
school, 

(10)  percent  of  teachers  with  master* 
degrees, 

(11)  percent  of  teachers  with  more  than 
3  years  experience,  and 

(12)  the  percent  of  teaching  days  that 
the  assigned  teachers  were  present 

in  the  classroom:  an  average  over  all  teachers 
assigned  to  the  school;  part-time  teachers  were 
prorated. 

These  were  available  measures  regularly  collected  and  under- 

7 

stood  by  the  school  administration.  They  were  aware  that  there  were 
other  desirable  input  measures,  but  chose  to  go  ahead  with  the 
analysis  to  gain  experience  with  the  method  which,  if  feasible  and 
useful,  would  be  repeated  with  more  carefully  designed  measures. 
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IV.  RESULTS 


4.1  Output  Tables  and  Interpretation 

For  the  Inputs  and  outputs  described  above,  we  calculated  the 
Inefficiency  rating,  z^»  for  167  elementary  schools  in  the  Houston 

Q 

Independent  School  District.  This  meant  solving  167  linear  programs 
each  having  14  rows  and  182  columns.  Of  these  schools,  46.7a  (78) 
were  found  to  be  inefficient.  In  order  to  display  both  efficiency 
and  effectiveness,  a  graph  relating  efficiency1 and  third  grade 
achievement  (see  Figure  1)  and  a  graph  relating  efficiency  and  sixth 
achievement  (see  Figure  2)  were  prepared.  In  order  to  present  complete 
results  and  preserve  the  readability  of  the  graphs,  the  following 
method  was  employed. 

1)  A  school’s  number  was  used  to  indicate  its 
associated  point  on  the  graph,  thus  providing 
identification  of  the  school  unit  along  with 
the  measurement. 

2)  Inefficient  schools  that  had  the  same  efficiency 
rating  and  output  measurement  as  some  other  school 
already  on  the  graph  were  listed  at  the  bottom  of 
the  figure, along  with  the  number  of  the  comparable 
school.  For  example,  schools  25  and  73  both  have 

h  *  .91  and  third  grade  achievement  of  3.5.  School 

It 

25  ia  shown  on  the  graph  in  Figure  1  and  73  is  paired 
with  25  at  the  bottom  of  Figure  1. 


Figures  1  and  2  About  Here 
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Schools  are  commonly  classified  as  above  or  below  soue 
achievement  norm.  However,  it  can  be  seen  from  Figures  1  and  2 
that  low  achieving  schools  may  be  efficient.  Consider  efficient 
school  87.  Given  its  very  low  outputs  (2.9  and  5.2),  it  might  seem 
that  this  school  was  not  doing  a  good  job.  However,  given  its  inputs, 
school  87  did  as  well  as  any  school  in  the  district;  much  better  than 
inefficient  school  49.  Even  though  these  two  schools  had  very  similar 
outputs,  one  was  efficient  and  the  other  inefficient.  From  Table  1,  it 
can  be  seen  that  school  49  did  not  fully  utilize  many  of  its  inputs,  the 
most  notable  being  $604/pupil  of  slack  local  and  state  funds.  Thus 
appropriate  planning  and  administrative  response  with  respect  to  these 
two  schools  would  be  quite  different. 

Note  that  relative  efficiency  is  displayed  in  Figures  1  and  2 
(horizontal  axis)  along  with  normed  measures  of  output  (vertical 
axis ) .  Thus,  both  effectiveness  and  efficiency  may  be  considered 
together.  No  measures  of  absolute  efficiency  are  provided., of  course, 
but  school  administrators  and  board  members  are  accustomed  to 
measuring  how  well  their  schools  perform  on  achievement  tests 
relative  to  a  norm.  With  the  additional  information  provided  by 
DEA,  they  are  able  to  assess  the  relationship  of  these  effectiveness 
indicators  to  the  quantity  of  resources  provided.  Thus,  a  school 
that  performs  poorly  with  respect  to  achievement  scores  would  be 
undesirable,  even  if  efficient,  but  the  management  response  would 
be  different  than  it  would  be  if  the  school  were  inefficient.  This 
'is  difccussed  mote' fully  in  the  following  section* 
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Table  1  About  Here 

The  information  provided  by  DEA  can  be  configured  in  many 
different  ways.  Since  space  is  limited,  it  was  decided  simply  to 
illustrate  by  including  two  possibilities9.  Table  1  focuses  on 
the  most  inefficient  schools  in  the  district.  All  these  schools 
appear  to  have  had  adequate  resources  if  fully  utilized10.  Table 
2  highlights  high-achieving,  near-efficient  schools.  It  might  be 
that  some  reallocatable  inputs  from  these  schools  could  be  trans¬ 
ferred  to  efficient  low-achieving  schools.  However,  such  reallocation 
decisions  involve  problems  of  relatiye  evaluation  and  causal  analyses 
that  go  beyond  the  scope  of  this  paper.  Hence  the  illustration  that 
follow  should  be  regarded  as  potential  extensions  that  could  flow 
naturally  from  a  DEA  analysis. 

Table  2  About  Here 

4,2  Administrative  Implications 

The  individual  school  is  the  production  unit  in  a  large  school  dis¬ 
trict.  The  General  Superintendent  and  his  central  staff  can  increase 
productivity  only  indirectly  —  through  hiring  and  assignment  of  per¬ 
sonnel  and  provision  of  resources  and  incentives  that  have  the  potential 
for  increasing  production  if  they  are  efficiently  employed.  As 
a  result,  Information  that  the  central  administration  may  use  to  guide 

system  planning  is  a  crucial,  and  often  missing,  factor  in  top-level 
management  of  schools. 

DEA  provides  this  management  information.  The  General  Superintendent 
and  his  staff  require  school  principals  to  submit  annual  operational  plans 


in  which  they  set  school  goals  and  specify  needed  resources.  These  opera¬ 
tional  plans  may  employ  DEA  results  as  follows:  (i)  as  information  for 
school  principals  to  use  in  their  proposed  plans,  (ii)  as  management "audit" 
information  for  use  in  reviewing  these  plans  before  approval,  (iii)  as  system 

scanning  information  to  balance  scarce  resources  among  schools,  and  (iv) 
for  annual  review  and  evaluation  of  accomplishment  of  the  previous  year's 
plan. 

In  developing  an  operating  plan  for  the  individual  school,  the 
principal  of  an  efficient  school  can  Increase  output  goals  if 
achievement  is  below  the  norm  and  will  be  able  to  request  additional 
resources  for  inputs  that  have  opportunity  costs.  The  principal 
of  an  inefficient  school  is  able  to  identify  those  Inputs  that  are 
unproductive.  Furthermore,  he  can  specify  vlthln-school  plans  for 
utilizing  inputs  more  effectively  in  order  to  increase  outputs  to 
the  level  of  expected  achievement  If  the  unit  were  efficient. 

For  example,  a  sample  planning  form  for  school  139  is  illustrated 
in  Display  1.  Planning  documents  of  this  type  have  proved  to  be  help¬ 
ful  with  respect  to  communication.  Using  Display  1,  the  principal 

nd 

would  observe  that  2  grade  achievement  in  1977  was  at  the  norm 
(2.8),  yet  these  same  children  a  year  later  1.  Ya  nrade  were  .5 
below  the  norm.  Further,  there  seems  to  be  a  cumulative  loss  in 
4th,  5th,  and  6**1  grades  as  5*^  is  1.2  below  the  norm  and  6^  is 
1.4.  Thus,  a  concentrated  effort  to  improve  achievement  in  the 
intermediate  grades  is  indicated;  that  is,  the  6^  grade  needs  to 
"catch  up".  School  faculty  should  examine  their  special  programs, 
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purchases  with  instructional  dollars  and  the  utilization  of  in¬ 
structional  personnel  as  the  display  indicates  that  these  resources 
are  not  being  effectively  used. 

Display  1  About  Here 

For  management  audit  of  proposed  plans,  the  General  Superintendent 
is  able  to  use  efficiency,  inefficiency,  slack,  and  opportunity  cost 
information  for  such  purposes  as:  determining  whether  low  achieve¬ 
ment  schools  are  setting  realistic  improvement  goals,  whether 
additional  input  request  are  justified,  and  what  inputs  are  not  being 
fully  utilized.  An  inefficient,  low  achieving  school  should  be  able 
to  achieve  outputs  specified  if  efficient  because  other  schools  in 
the  district  with  similar  inputs  were  able  to  achieve  such  outputs. 
Consequently,  requests  for  additional  input  would  not  seem  to  be 
justified.  On  the  other  hand  an  efficient,  low  achieving  school  could 
not  reasonably  be  expected  to  increase  its  output  unless  the  district 
provided  more  input  as  all  resources  were  used  as  effectively  as  in 
any  school  in  the  district.  Via  the  use  of  ranging  information  on 
the  right-hand-side  of  the  linear  program,  the  superintendent  can 
determine  the  amount  of  additional  inputs  which  can  be  productively 
used  under  the  current  production  rate.  Without  DEA  results,  such 
review  cannot  be  done  except  in  the  most  general  way  based  upon 
average  allocations  and  achievement  norms. 

As  a  means  of  balancing  resources,  th  central  staff  can 
identify  where  slack  resources  exist  in  Inefficient  schools  with 
high  output.  These  slack  resources  might  be  better  utilized  in 
efficient  schools  with  low  output.  For  example,  from  Figures  1  and 


rd 

2  we  find  that  school  8  is  efficient  but  low  achieving  (3  grade 

t*  h 

■  3.0,  6  grade  »  4.7)  while  school  63  is  inefficient  but  high 
achieving  (3rd  grade  *4.7,  6th  grade  ■  7.5),  From  Table  2,  we  see 
that  school  63  is  not  productively  utilizing  .7  professionals  per 
100  pupils,  $326  per  pupil  of  state  and  local  funds,  and  32.7%  of  their 
teachers  with  Masters  Degree.  Given  this  information,  the  General 
Superintendent  might  request  (1)  that  the  staff  of  School  8  submit 
plans  for  improving  achievement  along  with  specifications  of  additional 
input  requirements  needed  to  accomplish  these  goals  and  (2)  the  staff 
of  school  63  to  submit  plans  for  effectively  utilizing  their  non¬ 
productive  resources.  In  reviewing  these  plans,  some  of  the  non¬ 
productive  inputs  from  School  63  might  be  reallocated  to  School  8 
with  no  expected  loss  in  achievement  at  School  63  and  an  anticipated 
improvement  at  School  8.  A  subsequent  analysis  at  the  end  of  the 
next  school  year  would  allow  the  General  Superintendent  to  assess 
the  success  of  the  plans.  Of  course,  the  General  Superintendent 
would  want  to  consider  plans  from  all  the  schools,  not  just  two. 

The  above  comparison  of  two  schools  is  intended  to  illustrate  how 
DEA  results  could  be  used  for  balancing  resources. 

At  the  completion  of  each  year,  use  of  DEA  results  can  show 
whether  goal  achiev  ement  is  as  expected  and  whether  input  increments 
previously  authorized  were  efficiently  used.  An  overall  audit  of 
the  system  is  possible  to  determine  whether  the  proportion  of 
efficient  schools  is  increasing  or  decreasing. 
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Since  the  analysis  compares  each  school  to  the  observations 
obtained  on  all  other  schools,  it  follows  that  an  element  of 
competitiveness  is  involved.  If  an  efficient  school,  through 
better  operational  plans  succeeds  in  raising  its  achievement  more 
than  others,  then  some  schools  which  were  formerly  efficient  might 
become  inefficient.  Likewise,  some  inefficient  schools  might  be 
reduced  to  even  greater  inefficiency.  Competition  is  seen  to  be 
desirable  by  school  administrators  with  whom  we  have  worked  so 
long  as  it  does  not  unfairly  penalize  those  schools  which  have  many 
obstacles  to  overcome.  Fairness  of  results  la  evidenced  by  the 
number  of  efficient  schools  in  Figures  1  and  2  which  have  low 
achievement  because  of  high  proportions  of  children  from  disadvantaged 
families.  We  also  found  that  faculties  of  schools  who  had  been 
striving  for  better  results  were  pleased  to  have  an  objective 
measure  of  their  program. 
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V.  SUMMARY  AND  CONCLUSION 


The  General  Superintendent  of  Houston's  school  system  found  the 
pilot  testing  of  DEA  in  1979  to  be  sufficiently  interesting  so  that 
he  had  the  results  distributed  to  area  superintendents  and  to  school 
principals.  In  addition,  the  District  authorized  a  second  analysis 
using  1980  results  of  the  new  mandatory  statewide  competency  test  of  basic 
skills.  This  analysis  will  be  used  to  gain  additional  experience  with 
the  methodology. 

The  validity  of  DEA  was  assessed  in  an  informal  test  in  which  the 
General  Superintendent  and  his  staff  first  identified  "trouble" 
schools  and  "outstanding"  schools.  In  the  two  hour  session  approximately 
AO  of  the  167  schools  were  reviewed  in  detail  and  a  check  of 
administrative  assessment  against  DEA  solutions  resulted  in  100% 
correct  classification;  i.e.  the  "trruL  schools  were  all  inefficient 
to  some  degree  and  the  outstanding  schools  were  efficient.  Equally 
important,  the  reasons  for  a  school's  status  based  on  known  local 
conditions  generally  coincided  with  the  DEA  slack  values. 

The  three  major  problems  we  encountered  in  this  pilot  study  were 
(1)  obtaining  data  to  specify  adequate  input  measures,  (2)  obtaining 
data  to  specify  outputs  that  were  not  limited  to  cognitive  test  results, 
and  (3)  difficulties  in  communicating  the  results  of  a  complex 
quantitative  process  to  those  affected  by  the  results.  It  is  hoped 
that  these  problems  will  become  less  severe  with  increasing  experience. 

Other  applications  of  DEA  in  Texas  scfiools  are  currently  underway. 
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A  network  of  school  districts  has  recently  been  formed  with  a 
membership  of  235  elementary  schools  and  50  secondary  schools  in 
22  Texas  school  districts.  Known  as  the  Educational  Productivity 
Council,  the  schools  contribute  an  annual  membership  fee  to 
support  an  assessment  similar  to  the  one  reported  in  this  in¬ 
vestigation.  In  addition,  the  member  schools  will  cooperatively 
define  the  relevant  inputs  and  outputs  for  their  use  and  mutually 
share  in  efforts  to  develop  better  methods  of  application. 

At  another  level  of  application,  one  might  use  the  individual 
pupil  as  the  unit  of  analysis,  DEA  can  Identify  pupils  making  in¬ 
efficient  use  of  their  resources  for  learning;  this  information  can 
be  aggregated  at  the  classroom  level  to  measure  management  efficiency 
of  teachers  in  producing  a  learning  environment. 

This  methodology  is  appropriate  for  use  in  non-school 
applications  as  well.  For  example,  it  is  now  being  applied  to  high¬ 
way  safety  units.  Traffic  violations  and  convictions  are  being  used 
as  outputs  and  the  number  and  assignment  of  patrol  units  as  inputs. 
Other  potential  uses  for  not-for-profit  enterprises  include  the 
efficiency  analysis  of  welfare  units,  emergency  medical  service 
units,  and  post  office  mail  delivery. 
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Table  1  Schools  Less  Than  90%  Efficient 


School  Outputs 


Alterable  Slack  Resources  (Inputs) 


\ 

k 

3rd 

grade 

6  th 
grade 

Profess 

per 

100 

pupils 

Local  + 
State 
$ /pupil 

Fed  $ 

per 

pupil 

Stu. 

Attend¬ 

ance 

<%) 

Tea. 

Attend¬ 

ance 

(%) 

Tea. 

with 

Masters 

Degrees 

<%) 

Tea. 
with  > 

3  yrs. 
exp 
(%) 

11  Sp 

Prog 

.82 

23 

3.1 

5.2 

2.7 

$370 

$  44 

2.8% 

6.1% 

5.0% 

8.6% 

5.0 

.83 

12 

3.3 

5.7 

0 

$  24 

$  10 

0 

5.3% 

4.5% 

17.7% 

1.3 

.83 

30 

3.1 

5.3 

1.4 

$  28 

$  93 

4.0% 

0 

2.2% 

15.8% 

0 

.84 

i 

49 

3.0 

5.1 

4.0 

$604 

$107 

1.9% 

4.8% 

0 

0 

.5 

.85 

72 

3.1 

5.5 

.1 

0 

$  36 

3.4% 

2.2% 

9.6% 

o 

1.3 

.86 

139 

3.3 

5.4 

.5 

$  60 

$  82 

0 

1.5% 

8.9% 

18.8% 

1.5 

.86 

155 

3.2 

5.8 

0 

$  34 

0 

0 

3.2% 

0 

5.8% 

0 

.88 

116 

3.4 

5.2 

.1 

1 

$  74 

0 

6.4% 

3.9% 

23.4% 

17.1% 

3.1 

.88 

148 

3.8 

6.6 

1 

0 

0 

0 

1.4% 

0 

2.1% 

8.4% 

.3 

.89 

138 

3.5 

6.4 

0 

$145 

$  16 

0 

.8% 

0 

0 

0 

.89 

70 

3.2 

5.6 

.8 

0 

0 

4.9% 

5.4% 

0 

0 

.4 

.89 

44 

3.1 

5.2 

2.0 

$250 

$176 

4.0% 

7.6% 

0 

25.6% 

3.7 

.89 

165 

3.8 

6.5 

0 

0 

0 

.4% 

0 

4.5% 

29.2% 

0 

.89 

136 

3.7 

6.4 

.6 

$  70 

$  35 

1.5% 

3.4% 

11.5% 

0 

0 

Table  2  High-Achieving  Near-Efficient  Schools 
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Measurement  Observed  Measurement  Measurement 

Name _ Measurement _ if  Efficient _  Goal _ 


Output 

3rd  Grade 

(78) 

(norm-3.8)  3.3  Grade  Eauiv 

3.6 

6th  Grade 

(78) 

(norm-6.8)  5.4  Grade  Equiv 

6.3 

Input 

2nd  Grade 

(77) 

(norm-2.8)  2.8  Grade  Equiv 

2.7 

5th  Grade 

(77) 

(norm-5. 8)  4.6  Grade  Equiv 

4.5 

X  Non-minority 

1.7  X 

1.7 

X  Pay  lunch 

22.0  X 

22.0 

it  Prof./ 100 

5.9  prof/100  pupil 

5.4 

Federal  $/pupil  $210/pupil 

$128 

Local-State  $/pupil  $1185/pupil 

$1125 

%  Attendance 

90.8  X 

90.8 

%  Teacher 

days  worked  94.5  X 

93.0 

X  Masters  degree  44.0  % 

35.1 

%  More  3 

yrs. 

exp.  69.0  X 

50.2 

it  special  programs  6  special  programs 

4.5 

1 

•i 

1 


i 

\ 

'i 

■t 

1 

j 

i 
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FOOTNOTES 


1.  See  in  Levin  [23]  and  Charnes,  Cooper,  Rhodes  [llj  for  further 
discussion. 

2.  See  Charnes,  Cooper,  Rhodes  £l2j  . 

3.  See  Charnes,  Cooper,  Rhodes  [12J  . 

4.  We  also  do  not  use  the  change  of  variables  that  is  required  to 
relate  the  two  problems  to  each  other  but  rely  on  the  context 
to  make  clear  the  relations  betv/een  the  variables  in  (2)  and 
(3). 

5.  The  precise  conditions  for  optimality  and  their  relations 

to  the  non-Archemedian  £>0  values  are  set  forth  in  detail  in 

fio]  . 

6.  Note  that  inputs  are  defined  in  directionality  so  that  increasing 
units  of  input  would  be  expected  to  yield  increasing  units  of 
output . 

7.  Note  that  percentages  were  used  in  scaling  several  inputs  since 
in  each  case  the  measure  was  commonly  employed  in  that  form 

by  administrators.  As  Sherman  showed  in  [26]  ,  a  use  of  ratios 
may  produce  misclassifications.  This  was  checked,  however,  in 
a  variety  of  ways.  For  instance,  data  were  rescaled  as  log 
transforms  and  squared  proportions  and  there  were  no  signi¬ 
ficant  differences  in  computed  efficiency  index  values  or  in 
classification  of  units  as  efficient  or  inefficient.  Since 
the  ratio  approach  is  common  in  educational  management  and  since 
no  adverse  consequences  were  detected  in  the  above  described 
tests,  we  decided  to  continue  with  these  ratios  in  the  rest  of 
the  study. 


30 


8.  Only  elementary  schools  that  had  been  in  operation  more  than  two 
years  were  included. 

9.  For  other  variants,  see  Bessent  and  Bessent  [4]. 

10.  Inputs  such  as  minority  group  enrollment  and  socio-economic 
status  are  not  shown  in  the  tables  since  these  are  not 
alterable  by  management  decision  or  intervention.  They  may 
a£fect  the  efficiency,  however,  and  in  actual  practice  would 
be  displayed.  They  are  deleted  here  to  make  the  tables  more 
readable. 
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ITBS  Composite  Achievement 
for  3rd,  Grade _ i 


148,165 

136 


IV  Tm 


155  70 


I  5,117,134,135,147 
129  112,164 

17  1  14  18  64,77,82,103,105 


52,  24  1 15,  63,92,96,93 
151  7,  21,55 


,I40£8£9 


III 

>1  115,95 
5  54,150 
145 

- 66j46 

48,74 
!5  19 

50,131 


II  I  4,9.22,39.47,62,85 
65,68,99,160,162 
26,31,37,79 
101,107,109,141,144 

63,  3,  13,  20,  20,106, 120,130  | 

136, 41,56,57,58,69,84428,146,149 
I  2,43,59,91,94,97  j 

81,104,127  1 

45|  27,42,67,89,124,156  1 


I ncf f iciont  School 
not  on  Plot 


Sam«  os  School 
Numbtrtd 


EFFICIENCY  INDEX 


129  25  33  9<X8ttl53  61  80  II  12*119  I  20  44  113 


Fig.  I  Efficiency  and  Third  Grade  Achievement 


Unclassified 


ltcuwifv  classification  or  this  page  rwn*n  o«.  Kni«r*o 


REPORT  DOCUMENTATION  PAGE 

READ  INSTRUCTIONS 

BEFORE  COMPLETING  FORM 

1.  REPOAf  NUMltA  1.  OOVT  ACCEMION  NO. 

CCS  373  Ah-  A-^0  ' 

%.  RECIPIENT'S  CATALOO  NUMBER 

Iff . 

4.  TITLE  (ml  tnktltlo) 

An  Application  of  Mathematical  Prrgramming  to 
Assess  Managerial  Efficiency  in  the  Houston 
School  District 

S.  TYRE  OF  REPORT  A  PERIOD  COVERED 

4.  PERFORMING  ORG.  REPORT  NUMBER 

7.  author?*; 

A.  Bessent,  W.  Bessent,  J.  Kennington,  and 

B.  Reagan 

4.  46mtaacY  or  grant  numbers; 

N00014-81-C-0236 

N00014-75-C-0569 

»■  PERFORMING  ORGANIZATION  NAME  ANO  AOOREM 

10.  PROGRAM  ELEMENT.  PROJECT.  TASK 
AREA  «  PORK  UNIT  NUMBERS 

II.  CONTROLLING  OFFICE  NAME  ANO  AOOREM 

Office  of  Naval  Research  (Code  434) 

WashinGton.  D.  C.  .  . 

It.  REPORT  DATE 

November  1981 

IS-  NUMBER  OF  PAGES  /V, 

3ft  (J~)  >  / 

H.  MONITORING  AO*NCY  NAM*  *  ADORtSt <tt  Athrmt  tfm i  Co ntntUng  Ottif) 

SECURITY  CLASS,  (ol  I hit  nrort) 

Unclassified 

ISa.  OK£lAS9I  Ft  CATION/  DOWNGRADING 
SCHEDULE 

I*.  DISTRIBUTION  STATEMENT  (of  Oil.  Ktptii 


This  document  has  been  approved  for  public  release  and  sale;  its  distri¬ 
bution  is  unlimited. 


17.  DISTRIBUTION  STATEMENT  (of  Iho  oOltroct  OfttwW  In  atook  JO,  II  Mlhttnt  kom  R«#or 1) 


_ - _ ' 

is.  supplementary  notes  I 


IS.  KEY  WORDS  (Conllmto  *rt  tomotoo  olllo  II  noooooatj  onO 

lOonllU  Or  Olock  mmOot) 

|  Efficiency 

Production  Functions 

Linear  Programming 

Management  of  Schools 

Pareto -Koopmans  Optimality 

Estimation  Techniques 

Economic  Definitions 

Resource  Utilization 

SO.  ABSTRACT  (Contkmt  on  tomato*  olOo  II  MIIURI  aaW  tOoni Itr  kf  Oloot  l»wM 


A  school  may  be  viewed  as  an  enterprise  in  which  the  professsional  staff 
provide  the  operating  conditions  for  converting  quantifiable  resources  or 
inputs  into  pupil  learning  (outputs).  The  resources  are  determined  by 
budgets,  teacher  assignments,  and  student  assignments  while  learning  is 
determined  by  various  outputs  scored  according  to  standardized  tests  such 
as  the  Iowa  Test  of  Basic  Skills.  Following  the  work  of  Charnes,  Cooper 
and  Rhodes  [11],  we  use  a  ratio  definition  of  efficiency  which  takes 


no  FORM 
w  I  JAN  n 


147)  EDITION  OP  1  NOV  W  IB  OBSOLETE 
S/N  tlPt*tl4*SGSI  I 


_ Unclassified  . 

BECUR1TY  CLASSIFICATION  or  tnii  Mil 


I  VSStfKSm  MM  i 


■.llUNITy  CLASSIFICATION  OF  THIS  PM€Ctom  g «a  fcilH 


account  of  all  outputs  and  inputs  without  requiring  a  priori  specifi¬ 
cation  of  weights.  Instead  a  series  of  mathematical  programs  arc 
applied  to  determine  "virtual  multipliers"  from  actual  data  which  yield 
the  values  that  can  be  regarded  as  the  "most  favorable  weights"  for  each 
school  being  evaluated.  If  the  resulting  optimum  virtual  multipliers 
for  a  given  school  yields  an  efficiency  ratio  of  one,  then  that  school 
is  said  to  be  efficient.  If  the  ratio  is  less  than  one  then  that 
school  is  said  to  be  inefficient  relative  to  the  other  schools  in  the 
analysis.  The  ratio  is  also  accorded  operational  significance--it  is 
not  merely  an  index  number--  so  that  the  resulting  values  and  the  associ¬ 
ated  virtual  multipliers  make  it  possible  to  locate  where  improvements 
may  be  made  along  with  their  relative  magnitudes. 

This  analysis  was  applied  to  167  elementary  schools  in  the  Houston 
Independent  School  District.  Of  these  schools,  78  were  found  to  be  in¬ 
efficiently  utilizing  their  resources  as  compared  to  the  89  efficient 
schools.  Based  on  the  pilot  study,  an  Educational  Productivity  Council 
has  been  formed  at  the  University  of  Texas  at  Austin  to  provide  an 
annual  analysis  for  all  of  its  member  schools.  At  present  285 
Texas  schools  in  22  districts  are  scheduled  for  participation  in  the 
annual  analysis  as  described  in  this  investigation. 
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